Papers with selective SANs
How Does Selective Mechanism Improve Self-Attention Networks? (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show that selective SANs outperform the standard SAN by paying more attention to content words that contribute to the meaning of the sentence. |
| Approach: | They propose to implement selective SANs with a flexible Gumbel-Softmax to improve word order encoding and structure modeling. |
| Outcome: | The proposed system outperforms the standard SANs on several representative NLP tasks including natural language inference, semantic role labelling, and machine translation. |